Interrater Reliability Of Measurements Research Articles

Reliability of pediatric Rome IV criteria for the diagnosis of disorders of gut-brain interaction.

The diagnosis of disorders of gut-brain interaction (DGBI) in children is exclusively based on clinical criteria called the Rome criteria. The inter-rater reliability (IRR) measures how well two raters agree with a diagnosis using the same diagnostic tool. Previous versions of the Rome criteria showed only fair to moderate IRR. There have been no studies assessing the IRR of the current edition of the pediatric Rome criteria (Rome IV). This study sought to investigate the IRR of the pediatric Rome IV criteria and compare its reliability with the previous versions of the Rome criteria. We hypothesized that changes made to Rome IV would result in higher IRR than previous versions. This study used the same methodology as the previous studies on Rome II and III, including identical clinical vignettes, number of raters, and levels of expertise. Participants included 10 pediatric gastroenterology fellows and 10 pediatric gastroenterology specialists. IRR was assessed using the percentage of agreement and Cohen's kappa coefficient to account for possible agreement by chance. The average IRR percentage of agreement using the Rome IV criteria was 55% for pediatric gastroenterologists and 48.5% for fellows, indicating moderate agreement (k = 0.54 for specialists, k = 0.47 for fellows). The results demonstrated higher percentages of agreement and kappa coefficients compared to the Rome II and III criteria. The findings demonstrate improved reliability in Rome IV compared to Rome II and III, suggesting that the changes incorporated into the Rome IV criteria have enhanced diagnostic consistency. Despite the advancements, the reliability is still moderate, indicating the need for further refinement of future versions of the Rome criteria.

Intra- and Inter-Rater Reliability of Magnetic Resonance Imaging Measurements of Supraspinatus Muscle Thickness, Acromiohumeral Distance, and Coracohumeral Distance in Patients with Shoulder Pain

Aim: The aim of this study was to evaluate the intra- and inter-rater reliability coefficients of the supraspinatus muscle thickness (SMT ), acromiohumeral distance (AHD), and coracohumeral distance (CHD) in patients with shoulder pain. Materials and Methods: This retrospective study included 80 patients who presented with complaints of shoulder pain at a single centre between January 2023 and January 2024. The supraspinatus muscle thickness, acromiohumeral distance, and coracohumeral distance measurements were obtained one week apart by two observers. The intraclass correlation coefficient (ICC), minimum detectable change, and standard error of measurement were subsequently calculated. Results: AHD intra-rater reliability results evaluated on coronal sections were excellent (Rater 1, ICC=0.96; 95% CI, 0.94-0.97; Rater 2 ICC=0.75; 95% CI, 0.61-0.84). AHD intra-rater reliability results evaluated on sagittal sections were excellent (Rater 1, ICC=0.94; 95% CI; 0.91-0.96; Rater 2 ICC=0.77; 95% CI, 0.64-0.85). The results of CHD measured using coronal sections were good (Rater 1, ICC=0.85; 95% CI, 0.76-0.90; Rater 2 ICC=0.82; 95% CI, 0.71-0.88). SMT measurement results measured using coronal sections were excellent (Rater 1, ICC=0.98; CI, 0.98-0.99; Rater 2, ICC=0.89; 95% CI, 0.82-0.93).The inter-rater reliability values were good for AHD-Coronal (ICC= 0.75; 95% CI, 0.61-0.84) and AHD-Sagittal (ICC=0.86; 95% CI, 0.79-0.91), were fair for CHD evaluated on coronal sections (ICC=0.74; %95 CI, 0.58-0.83), and were excellent for SMT evaluated on coronal sections (ICC=0.92; 95% CI, 0.87-0.95). Conclusion: These results suggest that magnetic resonance imaging measurements of the supraspinatus muscle thickness and acromiohumeral distance can be reliable and consistent for the clinical evaluation of patients with shoulder pain. Keywords: Reliability, shoulder measurement, magnetic resonance imaging

A Dataset of Electrical Components for Mesh Segmentation and Computational Geometry Research

Data quality is of crucial importance in the field of automated or digitally assisted assembly. This paper presents a comprehensive data set of triangle meshes representing electrical and electronic components obtained by scraping Computer Aided Design (CAD) models from the Internet. Consisting of a total of 234 triangle meshes with labelled vertices, this data set was specifically created for segmentation tasks. Its versatility for multimodal tasks is underscored by the presence of various labels, including vertex labels, categories, and subcategories. This paper presents the data set and provides a thorough statistical analysis, including measures of shape, size, distribution, and inter-rater reliability. In addition, the paper suggests several approaches for using the data set, considering its multimodal characteristics. The data set and related findings presented in this paper are intended to encourage further research and advancement in the field of manufacturing automation, specifically spatial assembly.

Evaluation of the Interrater Reliability of Sonographic Measurements of Muscle Thickness of 38 Piriformis Muscles in 19 Patients with Piriformis Syndrome.

BACKGROUND The piriformis muscle is a flat superficial muscle of the deep gluteal muscles that externally rotates the hip. Ultrasound is widely used to identify the piriformis muscle, especially for guidance of the needle during injections; however, its diagnostic use has recently gained popularity. The operator-dependent nature of ultrasound requires demonstration of reliability between operators. This study aimed to evaluate interrater reliability of sonographic measurements of muscle thickness of 38 piriformis muscles in 19 patients with piriformis syndrome. MATERIAL AND METHODS An ultrasound transducer was placed transversely on the sacral spinous process and moved caudo-laterally until the piriformis muscle was visualized under the gluteus maximus while patients were lying in prone position. The thickness of piriformis muscle was measured with a 2 to 5-MHz broadband curvilinear transducer in 3 regions (thickest regions of muscle over the ilium, near the greater trochanter, and near the sacrum). The interrater reliability of measurements of 2 examiners who were blinded to each other's measurements was assessed by intraclass correlation coefficient. RESULTS In total, 114 samples from 38 piriformis muscles of 19 patients with a diagnosis of piriformis syndrome were evaluated by 2 raters in this study. The median (interquartile range) patient age was 41 (15) years. Intraclass correlation coefficient value for overall thickness measurements of piriformis muscle was 0.836. Intraclass correlation coefficient values for 3 different regions were over the ilium, near the greater trochanter, and near the sacrum were 0.777, 0.883, and 0.811, respectively. CONCLUSIONS Ultrasound measurement of piriformis muscle thickness has good interrater reliability.

Natural history of craniovertebral abnormalities in a single-center study in 54 patients with Hurler syndrome.

Craniovertebral junction (CVJ) abnormalities are common and well documented in mucopolysaccharidosis type I-Hurler syndrome (MPS IH), often causing severe spinal canal narrowing. However, the requirement for surgical decompression and/or fusion is uncommon. Although hematopoietic cell transplant (HCT) has been shown to prolong the lives of patients with MPS IH, its effect in halting or reversing musculoskeletal abnormalities is less clear. Unfortunately, there are currently no universal guidelines for imaging or indication for surgical interventions in these patients. The goal of this study was to track the progression of the CVJ anatomy in patients with MPS IH following HCT, and to examine radiographic features in patients who needed surgical intervention. Patients with MPS IH treated at the University of Minnesota with allogeneic HCT between 2008 and 2020 were retrospectively reviewed. Patients who underwent CVJ surgery were identified with chart review. All MPS IH cervical scans were examined, and the odontoid retroflexion angle, clivoaxial angle (CXA), canal width, and Grabb-Oakes distance (pB-C2) were measured yearly for up to 7 years after HCT. Longitudinal models based on the measurements were made. An intraclass correlation coefficient was used to measure interrater reliability. Nine children without MPS IH were examined for control CVJ measurements. A total of 253 cervical spine MRI scans were reviewed in 54 patients with MPS IH. Only 4 (7.4%) patients in the study cohort required surgery. Three of them had posterior fossa and C1 decompression, and 1 had a C1-2 fusion. There was no statistically significant difference in the spinal parameters that were examined between surgery and nonsurgery groups. Among the measurements, canal width and CXA varied drastically in patients with different neck positions. Odontoid retroflexion angle and CXA tended to decrease with age. Canal width and pB-C2 tended to increase with age. Based on the data, the authors observed an increase in canal width and pB-C2, whereas the CXA and odontoid retroflexion angle became more acute as the patients aged after HCT. The longitudinal models derived from these data mirrored the development in children without MPS IH. Spinal measurements obtained on MR images alone are not sufficient in identifying patients who require surgical intervention. Symptom monitoring and clinical examination, as well as pathological spinal cord changes on MRI, are more crucial in assessing the need for surgery than is obtaining serial imaging.

Improving the Objective Measurement of Alexithymia Using a Computer-Scored Alexithymia Provoked Response Questionnaire with an Online Sample

The study and measurement of alexithymia – a trait marked by difficulty identifying and describing feelings – can be improved by incorporating objective measures to supplement self-report scales. The Alexithymia Provoked Response Questionnaire (APRQ) is an observer-rated alexithymia tool that shows promise yet can be time-consuming to administer. The present study aimed to assess the feasibility of computer administration and scoring of the APRQ. Further, the APRQ’s association with verbal IQ and emotional vocabulary use was examined, as was the relationship between the APRQ and the self-report Bermond-Vorst Alexithymia Questionnaire-B (BVAQ-B). Adult participants (n = 366), including a proportion gathered through purposive sampling, participated in an online study. Inter-rater reliability measures indicated that computerized scoring of the APRQ is as reliable as human scoring, making the measure scalable for use with large samples. Alexithymia levels were independent of two measures of verbal IQ. Correlational analyses indicated overlap in alexithymia as measured by the APRQ and most of the subscales of the BVAQ-B. The APRQ, as an objective measure, may capture deficits in emotional awareness independent of self-insight.

Engaging classroom observation: A brief measure of active learning in the college classroom

The purpose of this study was to develop a valid, reliable, and brief measure of active learning in college classrooms that is cheap and easy to complete and yields results that faculty can easily use to inform their development as instructors. Initial construct and face validity was achieved by modifying existing instruments and creating a draft of a brief measure of active learning for external expert review. Following the suggested revisions, the engaging classroom observation was then piloted and revised as necessary. Reliability was tested and measures of internal consistency and interrater reliability were acceptable. A principal component analysis showed two components that were moderately correlated, which indicated the potential they could be combined. An Exploratory Factor Analysis confirmed the instrument is measuring one factor, which we propose as active learning. This study is significant because it offers a brief instrument based on students’ perceptions that can be used formatively by faculty.

Measuring Interrater Reliability

Measuring Interrater Reliability

Objective documentation of hypospadias anatomy with three-dimensional scanning

The absence of a standardized classification of hypospadias hinders understanding of the anatomic differences among patients and the evaluation of outcomes following surgical repair. In working towards a standardized, objective method of recording patients' hypospadias anatomy, we describe our initial experience using a non-invasive three-dimensional scanner. An Artec3D Space Spider scanner was used to obtain 3D scans in 29 patients undergoing hypospadias repair. Measurements of the urethral plate width, urethral plate length, glans width, penile shaft length, and penile shaft width were made by 2 pediatric urology attendings and 1 pediatric urology fellow. Measurements were compared and inter-rater reliability was calculated. A total of 435 measurements were made on 29 successfully generated 3D scans, ranging from distal to proximal hypospadias. The inter-rater reliability of measurements from the generated 3D models shown good inter-rater reliability of urethral plate width (ICC0.87 [95%CI:0.76,0.93]), penile shaft length (ICC0.87 [95%CI:0.70,0.94]) and glans width (ICC0.83 [95%CI:0.68,0.92]), excellent inter-rater reliability of urethral plate length (ICC0.96) and moderate inter-rater reliability of penile shaft width (ICC0.69 [95%CI:0.44,0.84]). There was a high degree of reliability of measurements made across multiple users. Calculation of the ratio of the urethral plate length/total penile shaft length objectively defined the initial position of the urethral meatus. When compared to the 3-dimensional volume of the glans, a more proximally positioned urethral meatus was associated with a lower glans volume. 3D scanning offers a rapid, reproducible, and non-invasive method of documenting hypospadias anatomy. The ability to evaluate three dimensional features (i.e. glans volume) offers an exciting opportunities for robust investigation of hypospadias outcomes and further understanding of the relationship between a patient's genotype and phenotype.

Feasibility of Ultrasound Measurements of Peripheral Sensory Nerves in Head and Neck Area in Healthy Subjects.

Current diagnostic methods for nerve compression headaches consist of diagnostic nerve blocks. A less-invasive method that can possibly aid in the diagnosis is ultrasound, by measuring the cross-sectional area (CSA) of the affected nerve. However, this technique has not been validated, and articles evaluating CSA measurements in the asymptomatic population are missing in the current literature. Therefore, the aim of this study was to determine the feasibility of ultrasound measurements of peripheral extracranial nerves in the head and neck area in asymptomatic individuals. The sensory nerves of the head and neck in healthy individuals were imaged by ultrasound. The CSA was measured at anatomical determined measurement sites for each nerve. To determine the feasibility of ultrasound measurements, the interrater reliability and the intrarater reliability were determined. In total, 60 healthy volunteers were included. We were able to image the nerves at nine of 11 measurement sites. The mean CSA of the frontal nerves ranged between 0.80 ± 0.42 mm2 and 1.20 ± 0.43 mm2, the mean CSA of the occipital nerves ranged between 2.90 ± 2.73 mm2 and 3.40 ± 1.91 mm2, and the mean CSA of the temporal nerves ranged between 0.92 ± 0.26 mm2 and 1.40 ± 1.11 mm2. The intrarater and interrater reliability of the CSA measurements was good (ICC: 0.75-0.78). Ultrasound is a feasible method to evaluate CSA measurements of peripheral extracranial nerves in the head and neck area. Further research should be done to evaluate the use of ultrasound as a diagnostic tool for nerve compression headache.

Intrarater and Interrater Reliability of Radiographic Characteristics in Skeletally Immature Patients With Anterior Cruciate Ligament Tears: A PLUTO Study Group Reliability Study.

Radiographic measurements of limb alignment in skeletally immature patients with anterior cruciate ligament (ACL) tears are frequently used for surgical decision-making, preoperative planning, and postoperative monitoring of skeletal growth. However, the interrater and intrarater reliability of these radiographic characteristics in this patient population is not well documented. Excellent reliability across 4 raters will be demonstrated for all digital measures of length, coronal plane joint orientation angles, mechanical axis, and tibial slope in skeletally immature patients with ACL tears. Cohort study (diagnosis). Three fellowship-trained orthopaedic surgeons and 1 medical student performed 2 rounds of radiographic measurements on digital imaging (lateral knee radiographs and long-leg radiographs) of skeletally immature patients with ACL tears. Intrarater and interrater reliability for continuous radiographic measurements was assessed with intraclass correlation coefficients (ICCs) across 4 raters with 95% CIs for affected and unaffected side measurements. Interrater reliability analysis used an ICC (2, 4) structure and intrarater reliability analysis used an ICC (2, 1) structure. A weighted kappa coefficient was calculated for ordinal variables along with 95% CIs for both interrater and intrarater reliability. Agreement statistic interpretations are based on scales described by Fleiss, and Cicchetti and Sparrow: <0.40, poor; 0.40 to 0.59, fair; 0.60 to 0.74, good; and >0.74, excellent. Radiographs from a convenience sample of 43 patients were included. Intrarater reliability was excellent for nearly all measurements and raters. Interrater reliability was also excellent for nearly all reads for all measurements. Radiographic reliability of long-leg radiographs and lateral knee x-rays in skeletally immature children with ACL tears is excellent across nearly all measures and raters and can be obtained and interpreted as reliable and reproducible means to measure limb length and alignment. Level III.

Creation of standardized tools to evaluate reporting in health research: Population Reporting Of Gender, Race, Ethnicity & Sex (PROGRES).

Despite increasing diversity in research recruitment, research finding reporting by gender, race, ethnicity, and sex has remained up to the discretion of authors. This study developped and piloted tools to standardize the inclusive reporting of gender, race, ethnicity, and sex in health research. A modified Delphi approach was used to develop standardized tools for the inclusive reporting of gender, race, ethnicity, and sex in health research. Health research, social epidemiology, sociology, and medical anthropology experts from 11 different universities participated in the Delphi process. The tools were pilot tested on 85 health research manuscripts in top health research journals to determine inter-rater reliability of the tools. The tools each spanned five dimensions for both sex and gender as well as race and ethnicity: Author inclusiveness, Participant inclusiveness, Nomenclature reporting, Descriptive reporting, and Outcomes reporting for each subpopulation. The sex and gender tool had a median score of 6 and a range of 1-15 out of 16 possible points. The percent agreement between reviewers piloting the sex and gender tool was 82%. The interrater reliability or average Cohen's Kappa was 0.54 with a standard deviation of 0.33 demonstrating moderate agreement. The race and ethnicity tool had a median score of 1 and a range of 0-15 out of 16 possible points. Race and ethnicity were both reported in only 25.8% of studies evaluated. Most studies that reported race reported only the largest subgroups; White, Black, and Latinx. The percent agreement between reviewers piloting the race and ethnicity tool was 84 and average Cohen's Kappa was 0.61 with a standard deviation of 0.38 demonstrating substantial agreement. While the overall dimension scores were low (indicating low inclusivity), the interrater reliability measures indicated moderate to substantial agreement for the respective tools. Efforts in recruitment alone will not provide more inclusive literature without improving reporting.

Comparison of Knee Articular Cartilage Defect Size Between Measurements Obtained on Preoperative MRI Versus During Arthrotomy.

Treatment decisions for cartilage defects are often based on lesion size. Magnetic resonance imaging (MRI) is widely used to diagnose cartilage defects noninvasively; however, their size estimated from MRI may differ from defect sizes measured during arthrotomy, especially after debridement to healthy cartilage if undergoing autologous chondrocyte implantation. The purpose of this study was to evaluate the reliability of 2 methods to assess knee cartilage defect size on preoperative MRI and determine their accuracy in predicting postdebridement defect sizes recorded during arthrotomy. It was hypothesized that defect size would be predicted more accurately by the total area of abnormal articular cartilage rather than the area of full-thickness cartilage loss as identified on MRI. Cohort study (diagnosis); Level of evidence, 3. This study included 64 patients (mean age, 41.8 ± 9.6 years) who underwent autologous cell therapy. Each patient received a 3-T MRI at 6.1 ± 3.0 weeks before cell implantation. Three raters, a radiologist, a surgeon, and a scientist, measured (1) the full-thickness cartilage defect area and (2) the total predicted abnormal cartilage area, identified by an abnormal signal on MRI. Interrater reliability was assessed using the intraclass correlation coefficient (ICC). Actual pre- and postdebridement defect sizes were obtained from intraoperative surgical notes. Postdebridement surgical measurements were considered the clinical reference standard and were compared with the radiologist's MRI measurements. Eighty-seven defects were assessed, located on the lateral (n = 8) and medial (n = 26) femoral condyle, trochlea (n = 17), and patella (n = 36). The interrater reliability of the cartilage defect measurements on MRI was good to excellent for the full-thickness cartilage defect area (ICC = 0.74) and the total predicted abnormal cartilage area (ICC = 0.78). The median full-thickness cartilage defect area on MRI underestimated the median postdebridement defect area by 78.3%, whereas the total predicted abnormal cartilage area measurement underestimated the postdebridement defect area by 14.3%. Measuring the full-thickness cartilage defect area on MRI underestimated the area to treat, whereas measuring the total abnormal area provided a better estimate of the actual defect size for treatment.

Longitudinal changes in qualitative aspects of semantic fluency in presymptomatic and prodromal genetic frontotemporal dementia.

The semantic fluency test is one of the most widely used neuropsychological tests in dementia diagnosis. Research utilizing the qualitative, psycholinguistic information embedded in its output is currently underexplored in presymptomatic and prodromal genetic FTD. Presymptomatic MAPT (n = 20) and GRN (n = 43) mutation carriers, and controls (n = 55) underwent up to 6years of neuropsychological assessment, including the semantic fluency test. Ten mutation carriers became symptomatic (phenoconverters). Total score and five qualitative fluency measures (lexical frequency, age of acquisition, number of clusters, cluster size, number of switches) were calculated. We used multilevel linear regression modeling to investigate longitudinal decline. We assessed the co-correlation of the qualitative measures at each time point with principal component analysis. We explored associations with cognitive decline and grey matter atrophy using partial correlations, and investigated classification abilities using binary logistic regression. The interrater reliability of the qualitative measures was good (ICC = 0.75-0.90). There was strong co-correlation between lexical frequency and age of acquisition, and between clustering and switching. At least 4years pre-phenoconversion, GRN phenoconverters had fewer but larger clusters (p < 0.001), and fewer switches (p = 0.004), correlating with lower executive function (r = 0.87-0.98). Fewer switches was predictive of phenoconversion, correctly classifying 90.3%. Starting at least 4years pre-phenoconversion, MAPT phenoconverters demonstrated an increase in lexical frequency (p = 0.009) and a decline in age of acquisition (p = 0.034), correlating with lower semantic processing (r = 0.90). Smaller cluster size was predictive of phenoconversion, correctly classifying 89.3%. Increase in lexical frequency and decline in age of acquisition were associated with grey matter volume loss of predominantly temporal areas, while decline in the number of clusters, cluster size, and switches correlated with grey matter volume loss of predominantly frontal areas. Qualitative aspects of semantic fluency could give insight into the underlying mechanisms as to why the "traditional" total score declines in the different FTD mutations. However, the qualitative measures currently demonstrate more fluctuation than the total score, the measure that seems to most reliably deteriorate with time. Replication in a larger sample of FTD phenoconverters is warranted to identify if qualitative measures could be sensitive cognitive biomarkers to identify and track mutation carriers converting to the symptomatic stage of FTD.

Functional measurement of canine muscular fitness: refinement and reliability of the Penn Vet Working Dog Center Sprint Test.

Working, sporting, and companion dogs require muscular fitness to perform their daily tasks, competitive activities, and operational functions effectively and with a low risk of injury. There are currently no methods to measure the muscular fitness of dogs who are not debilitated. Sprint performance is highly correlated with muscular fitness in humans, and various sprint assessments are used to measure performance for sporting and tactical athletes. The Penn Vet Working Dog Center Sprint Test (ST) is a 25 m maximal effort sprint from a down position and was developed to be a low-cost measure of muscular fitness suitable for field use. The purpose of this project was to describe the refinements to the ST, detail the performance and measurement protocol, evaluate the measurement inter-rater and intra-rater reliability, characterize the acceleration profile, and explore the inter-day reliability. Both naïve and experienced raters demonstrated excellent intra-rater and inter-rater reliability. The acceleration profile of the dogs in this study was similar to that of average adult human sprinters and demonstrated the role of muscular fitness in performance over this short distance. Finally, a small group of dogs showed moderate inter-day reliability and provided initial performance data to inform future studies. The ST appears to be a reliable measure of canine muscular fitness and could be used to assess performance in healthy dogs and guide the return to sport or work of debilitated dogs.

Intrareliability and Interreliability of Low-Cost Devices to Measure Skin Tone, Moisture, and Oiliness.

To evaluate the intraexaminer and interexaminer reliability of low-cost commercial devices to measure skin tone, moisture, and oiliness; determine associations with the Fitzpatrick Scale; and compare results with those of widely used commercial equipment. Researchers bilaterally collected a total of 36 samples from 18 participants. For data acquisition, two experienced raters were considered for skin index assessment. Evaluations were conducted independently, with measurements taken at two different times with an interval between them, thus enabling intrarater and interrater reliability measures. The measurements were made with two low-cost devices and compared with those acquired using standard equipment for such analyses. For the intraexaminer reliability results, the authors observed intraclass correlation coefficient ranging from moderate to high reliability between these tools (0.747-0.971). Regarding interexaminer reliability, intraclass correlation coefficient ranging from moderate to high (0.541-0.939) were observed. For the results of the correlations, a moderate to a large association was observed for skin tone. However, a small association for moisture was observed among the tools. Evaluations of skin tonality, oiliness, and moisture showed moderate to excellent intrareliability and interreliability. These methods can be applied in different environments, especially clinics, because of their low cost and ease of use.

Evaluation of inter-rater and test-retest reliability for near-infrared spectroscopy reactive hyperemia measures.

Near-infrared spectroscopy (NIRS) is a non-invasive tool used to measure blood flow in peripheral tissues. More information on inter-rater agreement and test-retest reliability of NIRS-based reperfusion assessments is needed. To assess inter-rater agreement for NIRS based data analysis, and evaluate the measurement's reliability across days. On three separate days (average days between visits 1 and 3: 19.4±6.9days), participants' (N=15 males, 22±2yr.) post-occlusion reactive hyperemia (PORH) was measured in the left gastrocnemius muscle using Continuous-Wave NIRS (CW-NIRS). A blood pressure cuff was placed proximal to the knee and inflated to occlude lower leg blood flow for 5min. The following CW-NIRS parameters were selected: (1) percent saturation in HbO2 (StO2%) at baseline; (2) the O2Hb range used to normalize the NIRS signal; (3) the time for the O2Hb signal to reach 50% peak post-occlusion hyperemia (T1/2), and (4) the post peak hyperemic O2Hb recovery slope (O2REC-SLP). Absolute agreement between the two analysts was calculated using two-way random effects Intraclass Correlation Coefficients (ICC2,1). Consistency between analysts and across days was calculated using two-way mixed models (ICC3,1). Mean and 95% confidence intervals (CI) of ICCs are reported. Coefficient of variation (CV) and standard error of the measurement (SEM) are reported. The ICC2,1 data indicated "adequate" to "excellent" absolute agreement between the two NIRS analysts. ICC2,3 data indicated "adequate" to "good" reliability across visits. The CV and SEM for rater 1 and rater 2 across visit were StO2 (CV: 3.79%±2.71% and 4.50%±2.37%; SEM: 3.42 and 3.82), O2Hb range (CV: 10.50±5.93 and 12.79±12.41; SEM: 3.26 and 4.71), T1/2 (CV: 11.15%±5.52% and 10.96%±4.50; SEM: 1.22 and 1.11), and O2REC-SLP (CV: 19.49%±9.99% and 18.45%±9.48%; SEM: 0.04 and 0.04). It is concluded that NIRS parameters assessed show adequate reliability between analysts and across three visits. It is recommended, when feasible and because of the absence of 100% reliability, that investigators employ more than one rater for scoring at least a portion of the data across each trial in a study's control condition in order to have the ability to estimate the magnitude of error attributable to imperfect reliability.

Reliability of Ultrasound Measurement of Plantar Fascia Thickness: A Systematic Review.

Because ultrasound measurement of plantar fascia thickness is widely used in the diagnosis and evaluation of plantar fasciitis, it is important to understand and minimize the errors that occur with this measurement. The aim of this systematic review was to identify and synthesize studies reporting on intrarater and interrater reliability of ultrasound measurement of plantar fascia thickness. After comprehensive searches in the MEDLINE, Embase, and Cochrane Library databases, 11 studies involving 238 healthy participants and 68 patients with pathologic foot disorders were included. Seven of 11 studies revealed a low risk of bias. Most of the studies reported good to excellent intrarater and interrater reliability for ultrasound measurement of plantar fascia thickness (intrarater intraclass correlation coefficient [ICC], 0.77-0.98; interrater ICC, 0.76-0.98). In addition, two studies on intrarater reliability and one study on interrater reliability showed moderate reliability (ICCs, 0.65, 0.67, and 0.59, respectively). Overall, the standard error of measurement was less than 5% and did not exceed 7%. The findings of this review suggest that ultrasound measurement of plantar fascia thickness is reliable in terms of both relative and absolute reliability. Reliability can be optimized by using the average of multiple measurements and an experienced operator.

Applying interrater reliability measure for user credibility assessment in reputation-oriented service discovery

Service Oriented Architecture is built on services in which web service discovery is one of the most widely explored domains. Service reputation plays a vital role in the discovery process while selecting the optimal service from a large pool of functionally equivalent services. Reputation mechanism attempts to forecast the future performance of a service based on its past behaviors generally obtained in the form of feedback ratings submitted by users of the service. These feedback ratings about a service may vary from user to user. Variations in feedback ratings could be because of different users’ different subjective judgments and/or dishonest users’ purposely submitted unfair ratings. This paper proposes a service reputation measurement approach counting such diversified act of rating. To realize this goal, an efficient user credibility assessment methodology has been devised by employing the measure of interrater reliability. Experiments are performed to validate the feasibility and effectiveness of the proposed reputation measurement approach. The experimental results depict that the proposed approach can fairly assess service reputations in the presence of various kinds of raters in the system.

Reliability and Agreement of Automated Head Measurements From 3-Dimensional Photogrammetry in Young Children.

This study aimed to assess the reliability and agreement of automated head measurements using 3-dimensional (3D) photogrammetry in young children. Specifically, the study evaluated the agreement between manual and automated occipitofrontal circumference (OFC) measurements (n = 264) obtained from 3D images of 188 patients diagnosed with sagittal synostosis using a novel automated method proposed in this study. In addition, the study aimed to determine the interrater and intrarater reliability of the automatically extracted OFC, cephalic index, and volume. The results of the study showed that the automated OFC measurements had an excellent agreement with manual measurements, with a very strong regression score (R2 = 0.969) and a small mean difference of -0.1 cm (-0.2%). The limits of agreement ranged from -0.93 to 0.74cm, falling within the reported limits of agreement for manual OFC measurements. High interrater and intrarater reliability of OFC, cephalic index, and volume measurements were also demonstrated. The proposed method for automated OFC measurements was found to be a reliable alternative to manual measurements, which may be particularly beneficial in young children who undergo 3D imaging in craniofacial centers as part of their treatment protocol and in research settings that require a reproducible and transparent pipeline for anthropometric measurements. The method has been incorporated into CraniumPy, an open-source tool for 3D image visualization, registration, and optimization, which is publicly available on GitHub (https://github.com/T-AbdelAlim/CraniumPy).